Modèle informatique du coapprentissage des ganglions de la base et du cortex : l'apprentissage par renforcement et le développement de représentations

Tout au long de la vie, le cerveau développe des représentations de son environnement permettant à l’individu d’en tirer meilleur profit. Comment ces représentations se développent-elles pendant la quête de récompenses demeure un mystère. Il est raisonnable de penser que le cortex est le siège de ces représentations et que les ganglions de la base jouent un rôle important dans la maximisation des récompenses. En particulier, les neurones dopaminergiques semblent coder un signal d’erreur de prédiction de récompense. Cette thèse étudie le problème en construisant, à l’aide de l’apprentissage machine, un modèle informatique intégrant de nombreuses évidences neurologiques. Après une introduction au cadre mathématique et à quelques algorithmes de l’apprentissage machine, un survol de l’apprentissage en psychologie et en neuroscience et une revue des modèles de l’apprentissage dans les ganglions de la base, la thèse comporte trois articles. Le premier montre qu’il est possible d’apprendre à maximiser ses récompenses tout en développant de meilleures représentations des entrées. Le second article porte sur l'important problème toujours non résolu de la représentation du temps. Il démontre qu’une représentation du temps peut être acquise automatiquement dans un réseau de neurones artificiels faisant office de mémoire de travail. La représentation développée par le modèle ressemble beaucoup à l’activité de neurones corticaux dans des tâches similaires. De plus, le modèle montre que l’utilisation du signal d’erreur de récompense peut accélérer la construction de ces représentations temporelles. Finalement, il montre qu’une telle représentation acquise automatiquement dans le cortex peut fournir l’information nécessaire aux ganglions de la base pour expliquer le signal dopaminergique. Enfin, le troisième article évalue le pouvoir explicatif et prédictif du modèle sur différentes situations comme la présence ou l’absence d’un stimulus (conditionnement classique ou de trace) pendant l’attente de la récompense. En plus de faire des prédictions très intéressantes en lien avec la littérature sur les intervalles de temps, l’article révèle certaines lacunes du modèle qui devront être améliorées. Bref, cette thèse étend les modèles actuels de l’apprentissage des ganglions de la base et du système dopaminergique au développement concurrent de représentations temporelles dans le cortex et aux interactions de ces deux structures.Throughout lifetime, the brain develops abstract representations of its environment that allow the individual to maximize his benefits. How these representations are developed while trying to acquire rewards remains a mystery. It is reasonable to assume that these representations arise in the cortex and that the basal ganglia are playing an important role in reward maximization. In particular, dopaminergic neurons appear to code a reward prediction error signal. This thesis studies the problem by constructing, using machine learning tools, a computational model that incorporates a number of relevant neurophysiological findings. After an introduction to the machine learning framework and to some of its algorithms, an overview of learning in psychology and neuroscience, and a review of models of learning in the basal ganglia, the thesis comprises three papers. The first article shows that it is possible to learn a better representation of the inputs while learning to maximize reward. The second paper addresses the important and still unresolved problem of the representation of time in the brain. The paper shows that a time representation can be acquired automatically in an artificial neural network acting like a working memory. The representation learned by the model closely resembles the activity of cortical neurons in similar tasks. Moreover, the model shows that the reward prediction error signal could accelerate the development of the temporal representation. Finally, it shows that if such a learned representation exists in the cortex, it could provide the necessary information to the basal ganglia to explain the dopaminergic signal. The third article evaluates the explanatory and predictive power of the model on the effects of differences in task conditions such as the presence or absence of a stimulus (classical versus trace conditioning) while waiting for the reward. Beyond making interesting predictions relevant to the timing literature, the paper reveals some shortcomings of the model that will need to be resolved. In summary, this thesis extends current models of reinforcement learning of the basal ganglia and the dopaminergic system to the concurrent development of representation in the cortex and to the interactions between these two regions

Double Counts in Aerial Surveys to Estimate Polar Bear Numbers During the Ice-Free Period

The double-count technique in aerial surveys, a variant of the mark and recapture method, was tested over islands offshore northern Quebec to estimate the number of polar bears that retreated there in the summers of 1986 and 1987. One front observer and two lateral ones surveyed six areas from aboard a twin engine DC-3 aircraft, independently reporting the number of animals they saw to the crew navigator. Bears were classified as being seen both in front and on the side, in front only or on the side only, making it possible to estimate correction factors. Although the observed strip covered 1.75 km on each side of the aircraft, the bear visibility rate exceeded 60% for lateral observers; the low vegetation of the islands and the contrasting colour of bears explain this high visibility. Corrected bear density varied between 0.4 and 14.2 animals per 100 sq km according to year and area. The double-count technique could be used to estimate the size of bear populations retreating on the islands and the coasts of Hudson Bay during the ice-free period, but its costs would have to be evaluated and compared with current techniques before including this method in management programs.Key words: polar bear, census, double count, Hudson Bay, Quebec, summer, Ursus maritimusMots clés: Baie d’Hudson, été, décompte double, inventaire, Québec, ours blanc, Ursus maritimu

CONTRIBUTION OF AUTHORS......................................................................VII

LITERATURE REVIEW........................................................................................

Knowledge transfer in neural networks : knowledge-based cascade-correlation

Most neural network learning algorithms cannot use knowledge other than what is provided in the training data. Initialized using random weights, they cannot use prior knowledge such as knowledge stored in previously trained networks. This manuscript thesis addresses this problem. It contains a literature review of the relevant static and constructive neural network learning algorithms and of the recent research on transfer of knowledge across neural networks. Manuscript 1 describes a new algorithm, named knowledge-based cascade-correlation (KBCC), which extends the cascade-correlation learning algorithm to allow it to use prior knowledge. This prior knowledge can be provided as, but is not limited to, previously trained neural networks. The manuscript also contains a set of experiments that shows how KBCC is able to reduce its learning time by automatically selecting the appropriate prior knowledge to reuse. Manuscript 2 shows how KBCC speeds up learning on a realistic large problem of vowel recognition

Combining TD-learning with cascade-correlation networks

Using neural networks to represent value functions in reinforcement learning algorithms often involves a lot of work in hand-crafting the network structure, and tuning the learning parameters. In this paper, we explore the potential of using constructive neural networks in reinforcement learning. Constructive neural network methods are appealing because they can build the network structure based on the data that needs to be represented. To our knowledge, such algorithms have not been used in reinforcement learning. A major issue is that constructive algorithms often work in batch mode, while many reinforcement learning algorithms work on-line. We use a cache to accumulate data, then use a variant of cascade correlation to update the value function. Preliminary results on the game of Tic-Tac-Toe show the potential of this new algorithm, compared to using static feed-forward neural networks trained with backpropagation. 1

Alternative Time Representation in Dopamine Models

Groupe de recherche sur le système nerveux central, Département d'informatique et de recherche opérationnelle, Département de physiologie.Données originales publiées dans l'article: Rivest, F, Kalaska, J.F., Bengio, Y. (2009) Alternative time representation in dopamine models. Journal of Computational Neuroscience. doi:10.1007/s10827-009-0191-1Instituts de recherche en santé du Canada, Fonds de la recherche en santé du Québec

Gagnon et al. Design and Validation of Emergency Management Solutions Using SYnRGY to Support Design and Validation Studies of Emergency Management Solutions

ABSTRACT Emergency management situations are highly complex and require the collaboration of multiple parties for adequate responses to incidents. The design and validation of effective emergency response systems is critical in order to improve the overall effectiveness of teams tasked to manage emergency situations. We report ongoing work whose objective is to increase the efficiency of emergency response solutions through iterative cycles of human in-the-loop simulation, modeling, and adaptation. Ultimately, this cycle could either be achieved offline for complex adaptation (e.g., development of a novel interface), or online to provide timely and accurate decision support during an emergency management event. The method is made possible by achieving a high degree of realism and experimental control through the use of an innovative emergency management simulation platform called SYnRGY

Access to Physician Services in Quebec: Relative Influence of Household Income and Area of Residence

The 1960s were marked by the setting up of Medicare by federal and provincial governments in Canada. The official aim of this undertaking was to eliminate inequality of access to medical care, and in the first instance inequality on the basis of incomes. Governments were also concerned with inequality in the regional distribution of medical services. The objective of this paper is to document these two dimensions of accessibility to medical services as present in Quebec in 1991 in terms of household consumption of such services. Data for this study come from administrative files, principally those of the Quebec ministry of health. The paper reviews measures taken by the Quebec government to attract doctors to locate in outlying regions of the province. In spite of these measures, results obtained indicate that significant differences existed in 1991 between outlying and central regions and that results for intermediate regions occur between these two poles. After controlling for age of head and household composition, differences by income are no longer significant.

Lecture et écriture 2.0 : utiliser les technologies Web pour lire et écrire /

Bibliogr. et webographie: p. [117]-122.Titre original: Literacy 2.0